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— The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36 (a). In no event, however, may a reply be timely filed after SIX (6) MONTHS from the 
mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timery filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 ) ix) Responsive to communication(s) filed on Jul 3, 2002 

2a) □ This action is FINAL. 2b) K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1 935 CD. 1 1 ; 453 O.G. 213. 
Disposition of Claims 

4) 5(1 Claim(s) 1-10 



4a) Of the above, claim(s) 
5)D Claim(s) 



6) 53 Claim(s) 1-10 

7) D Claim(s) 

8) D Claims 



is/are pending in the application, 
is/are withdrawn from consideration. 

is/are allowed. 

is/are rejected. 

is/are objected to. 



are subject to restriction and/or election requirement. 



Application Papers 

9) 

lOjVJ The drawing(s) filed on 



cted to 



9)D The specification is objected tq by the Examiner. 

ir 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 



is/are a) W accepted or b)D objected to by the Examiner. 



1 1 )□ The proposed drawing correction filed on 



is: a)D approved b)D disapproved by the Examiner. 



If approved, corrected drawings are required in reply to this Office action. 

1 2) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgement is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some* c)D None of: 

1. □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. 



3. □ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
*See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. § 1 19(e). 
a)D The translation of the foreign language provisional application has been received. 

15) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. §§120 and/or 121. 

Attachment(s) 

1 ) H Notice of References Cited (PTO-892) 4) □ Interview Summary (PT0-413) Paper No(s). 

21 ^Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) O Notice of Informal Patent Application (PTO-1 52) 

3) Q Information Disclosure Statement(s) (PTai 449) Paper No(s), 6) Q Other: 
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DETAILED ACTION 
Response to Amendment 

1. Applicants' arguments filed on July 3, 2002 ( paper # 4/A ) have been fully considered 
but are deemed to be moot in view of the new grounds of rejection. 

2. Applicant has submitted formal drawings which have been accepted. 

Claim Objections 

3. Claims 4 - 10 are objected to because of the following informalities: 
Claim 4 is objected to because of the following informalities: 

In claim 4, line 5, change " , " to 
In claim 4, line 7 , change " , " to -- ; - . 
Similarly all the claims need to be corrected . 
Appropriate correction is required. 

Claim Rejections - 35 U.S. C. § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103© 
and potential 35 U.S.C. 102(f) or (g) prior art under 35 U.S.C. 103(a). 

5. Claims 1, 4 -10, are rejected under 35 U.S.C. 103(a) as being unpatentable over Schuster 
etal., (US. 5,933,524), in view of Trew et al, (US. 5,280,530). 

As per claims 1 and 4, Schuster teaches a method of performing semi-automatic tracking 
of colored objects ( column 6, lines 28 - 29) within a video image sequence comprising the steps 
of ( abstract ) : 

separating objects within an initial frame ( note, segmentation of digital color images on 
the basis of color histogram which describe the characteristic color properties of the at least one 
imaged object for determining weather a pixel of a color image which is to be decompose into 
object segments belongs to a specific object segment or not , column 1, lines 45 - 62) of the 
video image sequence on the basis of color ( column 1, lines 49 - 59, column 2, lines 1 - 10) ; 

identifying from the separated objects an object of interest having a centroid ( note 
column 1, lines 49-59, column 3, lines 39-62, column 4, lines 38-65, and column 5, lines 26-56, 
note, that the centroid b, weighted average j^i and Si are the color moments) ; and 
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Schuster discloses method for segmentation of digital color images on the basis of color 
histograms . However, Schuster do not explicitly discloses tracking and a Kalman predictive 
algorithm applied to the centroid of an object. However, Trew discloses a method and apparatus 
for tracking a moving object . The system comprises of : 

tracking the object of interest through successive frames of the video image ( column 1, 
lines 27-60 ) sequence using a Kalman predictive algorithm applied to the centroid ( note, 
Kalman algorithm are applied to each pixel by taking a block of 3x3 pixels having the pixel of 
interest at its center ( column 4, lines 3- 1 1) , as shown by Trew the use of Kalman predictive 
algorithm applied to the centroid of an object because, by using a Kalman filtering process , the 
object appearance can be updated by incorporating the actual appearance ( column 4, lines 3-11 
). 

Therefore, it would have been obvious to one with ordinary skill in the art at the time of 
invention to incorporate the teaching as taught by Trew's into the system of Schuster, because, 
one with ordinary skill in the art would realize that using a Kalman filtering process , the object 
appearance can be updated by incorporating the actual appearance , as suggested by Trew at ( 
column 4, lines 3-11). 

As per claim 5, Trew teaches (New) a method according to claim 4, wherein step ( c ) 
includes determining the position of a centroid of the selected object and applying the Kalman 
predictive algorithm to the centroid ( column 4, lines 5- 26). 
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As per claim 6, Trew teaches (New) a method according to claim 4, wherein step ( c ) 
includes determining the position of a centroid based on a color function of the selected object 
and applying the Kalman predictive algorithm to the centroid ( column 4, lines 3-11, 25- 27 , 
column 11, lines 56- 61) . 

As per claim 7, Trew teaches (New) a method according to claim 4, wherein step © 
includes determining the position of a centroid based on luminance of the selected object and 
applying the Kalman predictive algorithm to the centroid ( column 3, lines 19- 27, column 5, 
lines 4- 28 ) . 

As per claim 8, Trew teaches (New) a method according to claim 4, wherein each image 
frame is resolved into multiple blocks and step (a) comprises segmenting the initial frame based 
on color of the blocks ( column 1 1, lines 49- 61). 

As per claim 9, Trew teaches (New) a method according to claim 8, wherein step (b) 
includes identifying a color model ( column 10, lines 51-60 ) to which the selected object 
belongs and step ( c ) includes: 

predicting the position of a centroid ( column 4, lines 25- 27 , column 5, lines 29- 37 ) , of 
the selected ( column 5, lines 22- 24 ) object in a subsequent frame ( column 4, lines 50-65) , 
determining whether the predicted position of the centroid in said subsequent frame is within a 
boundary of the selected object in said subsequent frame ( column 5, lines 29- 37, column 10, 
lines 25- 60 ) , 
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and, in the event that the predicted position of the centroid told in said subsequent frame 
is not within the boundary of the selected object in said subsequent frame ( column 10, lines 25 - 
60 ) , carrying out a search to identify a block that belongs to the selected color model ( column 
7, lines 15- 44). 

As per claim 10, Trew teaches (New) a method according to claim 4, wherein each 
image frame is resolved into multiple blocks and step ( c ) comprises: 

determining position and velocity of a centroid of the selected object in the initial frame ( 
note, updating the current picture or initial frame , column 5, lines 29- 37, column 10, lines 25- 
60), 

predicting ( column 3, lines 55 - 64 ) the position of the centroid in a subsequent frame ( 
column 4, lines 25- 27 , column 5, lines 29- 37 ), 

from the predicted position ( column 3, lines 61- 68 ) of the centroid in said subsequent 
frame ( column 4, lines 51- 65 ), extracting a connected group of blocks in said subsequent frame 
that belong to the selected object ( column 7, lines 15- 44 ) , and 

calculating the positional of the centroid of the selected object in said subsequent frame 
from the connected group of blocks ( column 5, lines 22- 44 ) . 

6. Claims 2 and 3, are rejected under 35 ILS.C. 103(a) as being unpatentable over Schuster 
et al., (US. 5,933,524), in view of Trew et ah, (US. 5,280,530), as applied to the above claim 1, 
4 -10 and further in view of Rangan et al, (US.6,198,833) . 
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Regarding claim 2, Schuster discloses method for segmentation of digital color images on 
the basis of color histograms . However, Schuster do not explicitly discloses velocity in 
determining the position of the frame . However, Rangan discloses a method for tracking a 
moving entity in a video presentation . The system comprises the step of : 

from the initial frame determining a position and velocity for the centroid (abstract, 
column 3, lines 36 - 38 ) ; 

for each successive frame predicting a position of the centroid ( note, correcting the 
position of the object on the frame, column 3, lines 36 - 38, column 4, lines 1-10 ); 

from the predicted position extracting a connected group of blocks that belong to the 
object of interest ( column 2, lines 38- 54 ); 

measuring the position of the centroid in the successive frame from the connected group 
of blocks ( note, measuring the position of moving entity in video by generating a matrix of 
signature pixels relative to the tracking object and the color values of each of the signature pixels 
in the first frame thereby creating a color signature for the object, testing the color signature at 
the plurality of test position in the immediate vicinity of the assumed position against the color 
signature recorded for the entity , accepting the position with the closest match to the color 
signature for the entity in which the process is repeated to determine positions for the entity in 
succeeding frames in the video data , column 2, lines 54-67) ; and 

smoothing ( column 11, lines 21-30) the measured position and velocity of the centroid ( 
column 9, lines 30-66, column 10, lines 1 -3 ), as shown by Rangan the use of velocity in 
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tracking a moving image by the use of vector . The process is thus contained from frame to frame 
, using the vector as a tool because this would improve a positioning and then testing for 
accuracy of the object ( column 10, lines 1 - 3) . 

Therefore, it would have been obvious to one with ordinary skill in the art at the time of 
invention that by incorporate the teaching as taught by Rangan's into the system of Schuster, 
because, one with ordinary skill in the art would realize that this would improved the position 
and accuracy of the object , as suggested by Rangan at ( column 10, lines 1 - 3) . 

As per claim 3, Rangan teaches the method as recited in claim 1 further comprising the 
steps of: 

detecting whether the centroid in the successive frame is within the object of interest and 
field of view ( column 9, lines 30-67) ; and 

applying an error recovery scheme to re-identify the object of interest in the successive 
frame ( column 10, lines 4-13). 
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Other prior art cited 

7. The prior art made of record and not relied upon is considered pertinent to applicants 
disclosure. 

Perona et al., (US. 6,044,165) discloses apparatus and method for tracking handwriting 
from visual input . 

Roehm et al., ( US. 5,293,574) discloses digital X-Ray imaging system with automatic 
tracking . 

Nielson ( US. 5,852,792) discloses spacecraft boresight calibration filter . 
Gray ( US. 5,05 1,751) discloses method of Kalman filtering for estimating the position 
and velocity of a tracked object . 
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Contact Information 



8. 



Any inquiry concerning this communication or earlier communications from the 



examiner should be directed to Sheela Chawan whose telephone number is (703) 305-4876. 
If attempts to reach the examiner on Monday through Thursday from 8:30 a.m. to 5: 00 p.m. by 
telephone are unsuccessful, the examiner's supervisor, Leo Boudreau , can be reached at (703) 
305- 4706. 

Any response to this action should be mailed to: 



or faxed to: 

(703) 872 - 9314, (for formal communications intended for entry) 
Or: Any inquiry of a general nature or relating to the status of this application should be 
directed to the Group Receptionist whose telephone number is (703)305-3900. 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 




Sheela Chawan 
Patent Examiner 
Group Art Unit 2621 
September 10, 2002 



Bhavesh Mehta 
Primary Examiner 



